Subject: DLSw Connectathon Draft Test Plan

From: [email-masked] (Scott Barvick)
Date: Wed, 31 May 1995 16:16:25 -0400 (EDT)

Everyone,

Attached is my first stab on a test plan for the DLSw Connectathon.
My goal is to get comments via email for the next few months and
then present a semi-final plan at the August AIW.  With the consensus
reached at the AIW, I will then finalize the plans and prepare for
October.

There are still many details to be worked out.  Hopefully this 
will give us enough information to get the right issues into
the open.  I'm sure that there will be many comments, and I
will try to satisfy as many as possible.  Larger issues
can be deferred until the August AIW.  

As an overall strategy, I thought that the APPN Connectathons
went very well so I tried to model as much as possible from it.  One 
area that appears to differ is the fact that an APPN
router essentially is one of the end stations.  With DLSw,
we need a whole lot more equipment to drive the SSP state
machines.  This adds to the logistics issues which we are working
on.

One note about test case #8.  I'm sure it will not pass without
comment.  I have received several requests for it to be 
included.  I support it because this is an opportunity to
do the testing/debugging that we all wish we could do, but 
never seem to have the proper forum for it.  The
fewer interoperability problems any DLSw product has, the
better for everyone.  

Again, the tests are intended to be suggestions.  Nobody
is required to run any tests that they are not interested in.

I had trouble with the -conformance mailing list so I
throwing it out to the general list.  Traffic is light anyway...

Thanks,
Scott
[email-masked]

------------------------------------------------------------------------

Goals
-----
To provide an environment for vendors to test interoperability 
of DLSw products.  The format should be sufficient to organize the
logistics of the testing while providing the opportunity for 
vendor-specific goals.  This is to be a strictly engineering
exercise to promote understanding of the DLSw protocol and 
foster interoperability among applications.

Non-Goals
---------
To provide the facilities for rigorous QA-style testing on
DLSw implementations.  This includes stress, scaling, or 
wide varieties of media.  For example, facilities to support
ethernet end stations will not be provided.

General Test Methodology: 
-------------------------
Each day vendors will be divided into groups of 4 and will spend the 
morning going through the desired elements of the suggested test plan.
Most of the tests are designed to operate between a pair of DLSw
implementations, but the initial groupings of four are intended to
provide more versatility in the testing and ability to continue
testing if one or more vendors experience problems.

The afternoon will be free form among the day's group to test
other features, work on problems from the morning, or expand
on morning tests.

Invitees
--------
All vendors with an implementation that supports
RFC 1795 (including Conformance Testing vendors).
Each vendor that operates as a DLSw switch (talks SSP across TCP/IP
on token ring) will have a bench and will participate in the
groups of 4.

Conformance test vendors which emulate a DLSw switch can participate
as one of the DLSw implementations in the groups of 4, or wait
for the free form or specifically arranged testing.

Vendors providing SNA, LLC, or NetBIOS testing emulation (no SSP),
will not be in the groups of 4.  Space for this equipment will
depend on lab real estate.  In order to provide control and consistency
for the base level testing, Bay Networks will not rely on outside
vendors to provide the equipment for the base testing.  These
types of test equipment will not be necessary for the base level
testing, but can be used as desired with the DLSw switch participants.

Commitment and non-refundable fee of $100 per company is required
by the end of AIW8.  The fee will be put towards the catered
lunches each day.

RSVPs are required by the end of AIW8 in August, 1995.  
Earlier, non-binding indications of participation would
be appreciated to make sure that the lab I reserve is large enough.

Bay Networks-provided facilities (tentative - for review only)
--------------------------------
1 bench per SSP vendor.  
Patch panel connections and MAUs to provide up to 20 token ring LANs.
Power
Data jacks
UTP cables to patch panels
IBM Mainframe connection.
2 NetBIOS PCs, 1 PU2.0, 1 PU2.1 PCs on token ring per group.
1 PU2.0 PC on SDLC per group.
Backbone IP token ring network.

Vendor-provided equipment (tentative - for review only)
-------------------------
Any datascopes required for debugging.
UTP transceivers for token ring operation (Comments???)
Any equipment required to install, manage, and debug implementation.

Suggested Test Items
--------------------
The following are some suggested test scenario recommendations.  These 
are all at your option - decide with each product you test against
which scenarios you will attempt.  Of course, you may expand on these
scenarios as you wish.  Space will be provided for you to track your
results.  In addition, you may use the Results Matrix (to be provided)
to "check off" your results with the other products.

1) Peer Connection Establishment and Capabilities Exchange
1.1 Test Steps
	a) Configure single remote DLSw switch and establish the
	pair of DLSw TCP connections.
	b) Verify TCP connection establishment and transfer of Capabilities
	Exchange messages.
	c) Change parameters to cause resend of Capabilities Exchange
	messages (optional).
	d) Break links and restart TCP connections, noting transfer
	of Capabilities messages each time.

2) SNA PU2.0 Circuit Establishment
2.1 Test Steps
	a) Bring up PU2.0 circuit between end station and host on token
	ring LANs.
	b) Break and restart circuit.
	c) Reverse location of end station and host and rerun a and b.

3) SNA PU2.1 Circuit Establishment
3.1 Test Steps
        a) Bring up PU2.1 circuit between end station and host on token
        ring LANs.
        b) Break and restart circuit.
        c) Reverse location of end station and host and rerun a and b.

4) NetBIOS Circuit Establishment
3.1 Test Steps
        a) Bring up NetBIOS circuit between end station and host on token
        ring LANs.
        b) Break and restart circuit.
        c) Reverse location of end stations and rerun a and b.

5) SNA PU2.0 SDLC Circuit Establishment
5.1 Test Steps
	a) Bring up PU2.0 circuit between SDLC end station and host on 
	token ring LAN.
        b) Break and restart circuit.

6) Flow Control
6.1 Test Steps
	a) Bring up PU2.0 circuit between end station and host on token
        ring LANs.
	b) Use IND$FILE to establish start data transfer from host to
	end station.
	c) Momentarily break link to token ring or IP cloud to cause flow
	control traffic shutdown.
	d) Verify reestablishment of smooth flow of data.
	e) Reverse location of end station and host and rerun a through d.

7) Multiple Peer Broadcasts
7.1 Test Steps
	a) Fully mesh DLSw peers in test group.
	b) Establish end station connection to host.  Note broadcast 
	distribution.
	c) Verify mac cache entry.
	d) Establish another end station connection to same host. 
	Verify cache use.
	e) Break end station connection and restart.  Verify correct
	broadcast distribution.
	f) Change location of end stations and rerun b through e.

8) RFC 1434 Interoperability
8.1 Test Steps
	a) Configure single remote DLSw switch and establish the pair
	of TCP connections.
	b) Verify transfer of Capabilities Exchange message to 1434 node.
	c) Run tests 3 through 5 as desired.

---

From: Dave Lowry <[email-masked]>
Date: Thu, 01 Jun 95 16:21:45 PDT

Scott,

First off, thank you for the effort you have put into coming up with
this initial draft. But, I have a few comments.

The main issue is with test case #8. As I recall, we discussed this
issue at the last DLSw RIG meeting. At that time there was consensus
that we would test RFC 1795 interoperabilty only. Now it seems you,
and others, want to change. I do not think it appropriate for us to
reach consensus at a meeting only to have that conclusion changed.  
Especially(and here's the hard part) when the subject is one which
could be considered biased toward a few vendors.

Even the RFC 1795 Spec. leads us to the belief that there is a
separation from RFC 1434. It is pretty clearly worded in section 
1.1 of the Spec. It states:

"This document defines significant changes to RFC 1434 and does not
state details on how to interoperate with RFC-1434 or "enhanced"
implementations (e.g. those that added enter and exit busy flow
control)."

I feel that adding RFC 1434 compatability testing to the DLSw RIG
sanctioned Connect-a-thon implies that RFC 1434 compatability was a
goal of the RFC 1795 effort. When in fact it was not.
The goal of the testing was for us all to have a chance to test
our RFC 1795 implementations for compatability with each of our peer's
implementations. Further, to debug if needed.

If several of our esteemed members feel it desirable to do backwards
compatability testing with their previous versions of code then they 
are free to do so. But the DLSw RIG sanctioned RFC 1795 connect-a-thon
is not the place to do it.

I move that test case #8 be removed. Please :-).

Dave

---

From: [email-masked] (Scott Barvick)
Date: Mon, 5 Jun 1995 11:48:57 -0400 (EDT)

Dave,
 
I understand your concern adding the RFC 1434/1795 testing after we said that
it would not be included.  Let me just add further comments to this
discussion and then pass responsibility for resolution of this issue to the
chairperson of this RIG, Ivan - I have already cleared this with him.
 
Basically, as I have stated before, I strongly believe that we should follow
the model of the APPN Connectathon in that the test cases are merely
suggestions.  No vendor is forced to do any of the tests if they don't
support a feature, if it is not ready for prime time yet, or any other reason. 
For instance, not everyone supports NetBIOS.  The suggestions are necessary to
establish enough order to the process so that we know what to prepare for and
what will happen once we begin testing.  Providing suggested test cases
allows time for vendors to decide which ones they would like to run.  We
observed this working quite well at the APPN Connectathon.  
 
As I worked on the test cases, I heard that several vendors were interested
in testing 1434 interoperation.  While I agree that the DLSw RIG is not
chartered to specify this function, I also believe that it is not precluded from
happening at the DLSw Connectathon.  Because several vendors are interested,
I thought it belonged in the suggested test cases in order to bring it
into the open so that everyone could opt not to do it or to prepare for 
it as they would any other test.
 
I do seem to be making some assumptions about agreement on the overall
strategy for the Connectathon.  If there is disagreement about these ideas, I
think we need to start there.  While we work on this issue, I hope 
that other comments about the goals, other test items, or procedures will
come out so that we all agree on how this event should happen.
 
Thanks,
Scott
[email-masked]

---

From: Dave Lowry <[email-masked]>
Date: Tue, 06 Jun 95 09:17:26 PDT

Scott,

I guess you and I can agree to disagree on whether having RFC1434
testing as a suggested test during the connect-a-thon implies
endorsement of backward compatability to that specific implementation
by the RIG. I think the best thing is if we do a poll of the rest of
the group as to who thinks it should be included and why.

I will make one final suggestion and then leave it to Ivan to try and 
get consensus on this. Talk about passing the buck ;-).

My feeling is that those who wish to do the RFC 1434 backward
compatibility testing can do so during the "off hours" of the
connect-a-thon. So you and the others who felt this important can
achieve your goal. But I still strongly feel that it should not be a
"suggested" test scenario. That gives the impression that the RIG is
endorsing RFC 1434. Which we have never done. I again move we delete
test #8.

I would really like to hear what others think!

Dave

---

From: [email-masked] (Michael Redden)
Date: Tue, 6 Jun 1995 14:17:03 -0400 (EDT)

Dave, Scott, Everyone,

Although I am much more interested in testing interoperability of the 1795 
implementation, I am also interested in testing interoperability with 
1434 implementations.  This type of testing should be wholly optional, but 
those of us who do wish to do this type of testing should be given the 
opportunity to do so.  

It seems to make a lot of sense to me to do 1434 testing at the same time as
1795 testing.  Interested parties would have already committed their time and
resources to the conformance testing effort.  I think it would be rather 
foolish to require 1434 testing at a different time when the people and 
equipment will be already there for 1795 testing.

Mike Redden
Proteon